Apparatus for embedding and reproducing watermark into and from contents data

ABSTRACT

An apparatus for embedding a watermark into contents data includes a parameter converting device for converting a parameter of first contents data to generate second contents data. A mixing device operates for embedding parameter information into the second contents data as watermark information. The parameter information represents a condition of the conversion of the parameter by the parameter converting device. The watermark information may include copyright information.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to an apparatus for embedding or inserting awatermark into contents data such as picture data, audio data, audiovisual data, or multimedia data. In addition, this invention relates toan apparatus for reproducing or extracting a watermark from watermarkedcontents data. Furthermore, this invention relates to a recording mediumfor storing watermarked contents data.

2. Description of the Related Art

H. Ogawa et al. have reported “A Copying Information Embedding Methodusing DCT for Digital Movies”, SCIS'97-31G, which discloses watermarkingmethods suitable for MPEG bit streams. The watermarking methods arebased on modifying DCT coefficients, motion vectors, and quantizermatrices. Regarding contents data into which watermark information hasbeen inserted by the DCT-coefficient-modifying watermarking methodreported by H. Ogawa et al., the accuracy of reproducing the watermarkinformation from the contents data is considerably low when thesynchronization with DCT-sampling units (DCT blocks) fails to beacquired. The failure of the acquisition of the synchronization causesDCT blocks to shift from true positions by, for example, one pixel orone line in a frame. It is assumed that picture data which have beenwatermarked by one of the methods reported by H. Ogawa et al. aresubjected to affine transformation to slightly rotate or distort apicture. It tends to be difficult to accurately extract the watermarkfrom the transformation-resultant watermarked picture data.

J. Ohnishi et al. have reported “A Watermarking Scheme to Image Data byPN Sequence”, SCIS'97-26B, which discloses a data hiding method using aPN sequence in the spread spectrum technique. In the data hiding method,an original image signal is converted into a spread spectrum in responseto a PN sequence. A narrow band signal to stand for a signature is addedto the spread spectrum, that is, a wideband channel of which an originalimage is spread. When the signature-added spread spectrum is inverselyconverted into the normal image by the PN sequence, the signature signalis spread over the normal-image signal. In other words, the signaturesignal is embedded in the normal-image signal. The spread signaturesignal is low in power, and hence hardly acts as noise with respect tothe original image. Accordingly, the signature-added image issubstantially the same as the original image. When the signature-addedimage signal is spread by the PN sequence, the signature signal isreproduced.

SUMMARY OF THE INVENTION

It is a first object of this invention to provide an improved apparatusfor embedding a watermark into contents data.

It is a second object of this invention to provide an improved apparatusfor reproducing a watermark from watermarked contents data.

It is a third object of this invention to provide an improved recordingmedium for storing watermarked contents data.

A first aspect of this invention provides an apparatus for embedding awatermark into contents data. The apparatus comprises parameterconverting means for converting a parameter of first contents data togenerate second contents data; and mixing means for embedding parameterinformation into the second contents data as watermark information, theparameter information representing a condition of the conversion of theparameter by the parameter converting means.

A second aspect of this invention is based on the first aspect thereof,and provides an apparatus wherein the parameter converting meanscomprises means for converting a parameter of segments of the firstcontents data which correspond to pixels forming a specified pictureportion.

A third aspect of this invention is based on the first aspect thereof,and provides an apparatus wherein the mixing means comprises means forembedding copyright information and the parameter information into thesecond contents data as watermark information.

A fourth aspect of this invention is based on the first aspect thereof,and provides an apparatus wherein the parameter converting meanscomprises means for converting a parameter of segments of the firstcontents data which correspond to pixels at watermark-embeddedpositions, and the parameter information includes a parameter valueindicative of a rate of the conversion of the parameter, wherein themixing means comprises a first mixer and a second mixer, wherein thefirst mixer comprises pattern generating means for generating bitsrepresenting a predetermined bit pattern, specified-bit detecting meansfor detecting bits in the second contents data as specified bits whichcorrespond to the pixels at the watermark-embedded positions,calculating means for calculating a desired bit pattern represented bythe specified bits in response to the predetermined bit pattern and aspecified bit pattern, and changing means for changing the specifiedbits to represent the desired bit pattern to convert the second contentsdata into bit-pattern-added contents data, and wherein the second mixercomprises means for embedding copyright information and the parameterinformation into the bit-pattern-added contents data as watermarkinformation.

A fifth aspect of this invention provides an apparatus for reproducing awatermark from watermarked contents data generated by converting aparameter of original contents data to get conversion-resultant originaldata and embedding parameter information into the conversion-resultantoriginal data as watermark information, the parameter informationrepresenting a condition of the conversion of the parameter. Theapparatus comprises parameter detecting means for detecting theparameter information from the watermarked contents data; and parameterinversely converting means for inversely converting the watermarkedcontents data into the original contents data in response to theparameter information detected by the parameter detecting means.

A sixth aspect of this invention is based on the fifth aspect thereof,and provides an apparatus wherein the watermarked contents data includecopyright information and the parameter information as the watermarkinformation, and further comprising copyright information detectingmeans for detecting the copyright information from the watermarkedcontents data.

A seventh aspect of this invention is based on the fifth aspect thereof,and provides an apparatus wherein the watermarked contents data havebeen generated by converting a parameter of segments of the originalcontents data which correspond to pixels at watermark-embeddedpositions, and the parameter information includes a parameter valueindicative of a ratio of the conversion of the parameter, wherein theparameter detecting means comprises pattern generating means forgenerating bits representing a predetermined bit pattern, operationmeans for selecting specified bits among bits in the watermarkedcontents data, for repetitively changing the currently-selectedspecified bits from ones to others, and for executing given logicaloperation between the predetermined bit pattern and a bit patternrepresented by the currently-selected specified bits, embedding-positiondetecting means for deciding whether or not a result of the givenlogical operation is equal to a specified bit pattern, and for, when theresult of the given logical operation is equal to the specified bitpattern, deciding that the currently-selected specified bits correspondto a watermark-embedded position, and parameter-value detecting meansfor detecting the parameter value in the detected parameter information,wherein the parameter inversely converting means comprises an inverseconverter for, in response to the parameter value detected by theparameter-value detecting means, inversely converting the parameter ofthe segments of the watermarked contents data which correspond to thepixels at the watermark-embedded positions decided by theembedding-position detecting means.

An eighth aspect of this invention is based on the seventh aspectthereof, and provides an apparatus wherein the predetermined bit patternand the specified bit pattern remain unchanged when being rotatedthrough one of 90, 180, and 270 degrees.

An eighth aspect of this invention provides a recording medium forstoring watermarked data including contents data and watermarkinformation, the contents data having a parameter converted from anoriginal value, the watermark information being embedded in the contentsdata, the watermark information including parameter informationrepresenting a condition of the conversion of the parameter from theoriginal value.

A tenth aspect of this invention is based on the ninth aspect thereof,and provides a recording medium wherein a parameter of segments of thecontents data which correspond to pixels at watermark-embedded positionsis converted from an original value, and the parameter informationincludes a parameter value indicative of a rate of the conversion of theparameter, wherein the watermarked data are produced by generating bitsrepresenting a predetermined bit pattern, detecting bits in the contentsdata as specified bits which correspond to the pixels at thewatermark-embedded positions, calculating a desired bit patternrepresented by the specified bits in response to the predetermined bitpattern and a specified bit pattern, changing the specified bits torepresent the desired bit pattern to convert the contents data intobit-pattern-added contents data, and embedding the parameter informationinto the bit-pattern-added contents data as watermark information.

An eleventh aspect of this invention provides a recording medium forstoring watermarked data including contents data and watermarkinformation, the contents data having a parameter converted from anoriginal value, the watermark information being embedded in the contentsdata, the watermark information including copyright information andparameter information representing a condition of the conversion of theparameter from the original value.

A twelfth aspect of this invention provides an apparatus for embedding awatermark into contents data. The apparatus comprises address generatingmeans for generating a jump-destination address; information generatingmeans for generating copyright information; mixing means for embeddingthe jump-destination address generated by the address generating meansand the copyright information generated by the information generatingmeans into input contents data as watermark information to generatefirst watermark-added contents data; an encoder for compressivelyencoding the first watermark-added contents data generated by the mixingmeans into second watermark-added contents data; and rearranging meansfor rearranging unit portions of the second watermark-added contentsdata generated by the encoder in response to the jump-destinationaddress generated by the address generating means, wherein the unitportions are defined by the encoding by the encoder.

A thirteenth aspect of this invention provides an apparatus forembedding a watermark into contents data. The apparatus comprisesaddress generating means for generating a jump-destination address;information generating means for generating copyright information;mixing means for embedding the jump-destination address generated by theaddress generating means and the copyright information generated by theinformation generating means into input contents data as watermarkinformation to generate watermark-added contents data; and rearrangingmeans for rearranging unit portions of the watermark-added contents datagenerated by the mixing means in response to the jump-destinationaddress generated by the address generating means, wherein the unitportions represent respective divided regions composing a still-pictureframe.

A fourteenth aspect of this invention provides an apparatus forreproducing a watermark from watermarked contents data. The apparatuscomprises rearranging means for rearranging unit portions of firstwatermark-added contents data in an original order to generate secondwatermark-added contents data in response to a jump-destination address;watermark information detecting means for detecting watermarkinformation from the second watermark-added contents information;address calculating means for calculating the jump-destination addressfrom a related portion of the watermark information detected by thewatermark information detecting means, and for notifying the calculatedjump-destination address to the rearranging means; a decoder fordecoding the second watermark-added contents; and copyright informationdetecting means for detecting copyright information from the watermarkinformation detected by the watermark information detecting means.

A fifteenth aspect of this invention provides an apparatus forreproducing a watermark from watermarked contents data. The apparatuscomprises rearranging means for rearranging unit portions of firstwatermark-added contents data in an original order to generate secondwatermark-added contents data in response to a jump-destination address,wherein the unit portions represent respective divided regions composinga still-picture frame; watermark information detecting means fordetecting watermark information from the second watermark-added contentsinformation; address calculating means for calculating thejump-destination address from a related portion of the watermarkinformation detected by the watermark information detecting means, andfor notifying the calculated jump-destination address to the rearrangingmeans; and copyright information detecting means for detecting copyrightinformation from the watermark information detected by the watermarkinformation detecting means.

A sixteenth aspect of this invention provides a recording medium forstoring watermarked data including contents data and watermarkinformation, the contents data resulting from a prescribed encodingprocedure, the watermark information being embedded in the contentsdata, the watermark information including a jump-destination address andcopyright information, the contents data having unit portions arrangedin a scrambled order different from an original order, the unit portionsbeing defined by the prescribed encoding procedure, the jump-destinationaddress being for enabling the unit portions to be rearranged in theoriginal order.

A seventeenth aspect of this invention provides a recording medium forstoring watermarked data including contents data and watermarkinformation, the contents data representing a still picture, thewatermark information being embedded in the contents data, the watermarkinformation including a jump-destination address and copyrightinformation, the contents data having unit portions arranged in ascrambled order different from an original order and representingrespective divided regions composing the still picture, thejump-destination address being for enabling the unit portions to berearranged in the original order.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an apparatus for embedding a watermark intocontents data according to a first embodiment of this invention.

FIG. 2 is a block diagram of a sub-mixer in FIG. 1.

FIG. 3 is a diagram of a 1-frame-corresponding picture represented by aluminance signal Y, a 1-frame-corresponding picture represented by afirst color difference signal Cb, and a 1-frame-corresponding picturerepresented by a second color difference signal Cr.

FIG. 4 is a diagram of an example of a watermark.

FIG. 5 is a diagram of 1-pixel-corresponding 8-bit segments of aluminance signal in which only the lowest bit among 8 bits per luminancepixel is selected as a bit into which a watermark fragment is embedded.

FIG. 6 is a diagram of 1-pixel-corresponding 8-bit segments of aluminance signal in which only the two lower bits among 8 bits perluminance pixel is selected as bits into which a watermark fragment isembedded.

FIG. 7 is a diagram of 1-pixel-corresponding 8-bit segments of aluminance signal in which only the three lower bits among 8 bits perluminance pixel is selected as bits into which a watermark fragment isembedded.

FIG. 8 is a diagram of an example of a fixed 3-by-3-bit pattern producedby a fixed-pattern generator in FIG. 2.

FIG. 9 is a diagram of a specified 3-by-3-bit pattern used in theapparatus of FIG. 1.

FIG. 10 is a block diagram of an apparatus for reproducing a watermarkfrom watermarked contents data according to the first embodiment of thisinvention.

FIG. 11 is a block diagram of a watermark detector in FIG. 10.

FIG. 12 is a block diagram of an apparatus for embedding a watermarkinto contents data according to a second embodiment of this invention.

FIG. 13 is a block diagram of an apparatus for reproducing a watermarkfrom watermarked contents data according to the second embodiment ofthis invention.

FIG. 14 is a block diagram of an apparatus for embedding a watermarkinto contents data according to a sixth embodiment of this invention.

FIG. 15 is a time-domain diagram of a sequence of cells in an originalplayback order.

FIG. 16 is a time-domain diagram of a first sequence of cells in anorder different from an original playback order.

FIG. 17 is a time-domain diagram of a second sequence of cells in anorder different from an original playback order.

FIG. 18 is a block diagram of an apparatus for reproducing a watermarkfrom watermarked contents data according to the sixth embodiment of thisinvention.

FIG. 19 is a diagram of sub-regions composing a frame and being arrangedin a raster scanning order.

FIG. 20 is a diagram of sub-regions composing a frame and being arrangedin an order different from a raster scanning order.

FIG. 21 is a diagram of sub-regions composing a frame and being arrangedin a raster scanning order.

FIG. 22 is a diagram of sub-regions composing a frame and being arrangedin an order different from a raster scanning order.

FIG. 23 is a block diagram of an apparatus for embedding a watermarkinto contents data according to a ninth embodiment of this invention.

FIG. 24 is a block diagram of an apparatus for reproducing a watermarkfrom watermarked contents data according to the ninth embodiment of thisinvention.

DETAILED DESCRIPTION OF THE INVENTION First Embodiment

FIG. 1 shows an apparatus for embedding or inserting a watermark intocontents data according to a first embodiment of this invention. Theapparatus of FIG. 1 includes a parameter converter 11, a watermark mixer12, a parameter generator 13, a copyright information generator 14, anMPEG encoder 15, and a data writer 16. Here, “MPEG” is short for “MovingPicture Experts Group”. The parameter converter 11 has acolor-difference-signal DC offset converter 21 and an embedding-positiondetector 22. Here, “DC” is short for “direct current”. The watermarkmixer 12 has a first sub-mixer 24 and a second sub-mixer 25. Theparameter generator 13 has a DC offset generator 23.

The color-difference-signal DC offset converter 21 is connected with theembedding-position detector 22, the DC offset generator 23, the firstsub-mixer 24, and the second sub-mixer 25. The embedding-positiondetector 22 is connected with the first sub-mixer 24. The DC offsetgenerator 23 is connected with the second sub-mixer 25. The firstsub-mixer 24 is connected with the second sub-mixer 25. The secondsub-mixer 25 is connected with the copyright information generator 14and the MPEG encoder 15. The MPEG encoder 15 is connected with the datawriter 16.

As shown in FIG. 2, the first sub-mixer 24 has a pixel specified-bitdetector 27, a fixed-pattern generator 28, and an exclusive-OR device29. The pixel specified-bit detector 27 is connected with thecolor-difference-signal DC offset converter 21 (see FIG. 1) and theexclusive-OR device 29. The fixed-pattern generator 28 is connected withthe embedding-position detector 22 (see FIG. 1) and the exclusive-ORdevice 29. The exclusive-OR device 29 is connected with the secondsub-mixer 25 (see FIG. 1).

Input contents data representing an original picture or a sequence oforiginal pictures are fed to the color-difference-signal DC offsetconverter 21 and the embedding-position detector 22. The originalpicture data (the input contents data) are composed of a luminancesignal Y and color difference signals Cb and Cr.

As shown in FIG. 3, one picture (one frame) represented by the luminancesignal Y has 720 pixels in a horizontal direction, and 480 lines in avertical direction. One picture (one frame) represented by the colordifference signal Cb has 360 pixels in the horizontal direction, and 480lines in the vertical direction. Similarly, one picture (one frame)represented by the color difference signal Cr has 360 pixels in thehorizontal direction, and 480 lines in the vertical direction. Each ofthe luminance signal Y and the color difference signals Cb and Cr has 8bits per pixel.

Map data representing a watermark are fed to the embedding-positiondetector 22. FIG. 4 shows an example of the watermark, “JVC”, which isrepresented by the map data. A frame represented by the map data iscomposed of divided rectangular blocks each having, for example, 81neighboring luminance pixels being 9 luminance pixels in a horizontaldirection by 9 luminance pixels in a vertical direction. Prescribedblocks D selected among the divided rectangular blocks represent thewatermark, “JVC”. The map data indicate the positions of the prescribedblocks D with respect to the frame. The positions of the prescribedblocks D correspond to watermark-embedding positions.

The watermark map data may be stored in a memory. In this case, thewatermark map data are fed from the memory to the embedding-positiondetector 22. The watermark map data may be fed to the embedding-positiondetector 22 from an external.

The embedding-position detector 22 derives the watermark-embeddingpositions from the watermark map data. The embedding-position detector22 converts the watermark-embedding positions into addresses(watermark-embedding addresses) defined in the case where the prescribedblocks D are scanned in a raster scanning order. The embedding-positiondetector 22 notifies the watermark-embedding addresses to thecolor-difference-signal DC offset converter 21 and the fixed-patterngenerator 28 in the first sub-mixer 24.

The DC offset generator 23 produces an 8-bit signal representing a DCoffset value which can change from −128 to +127 in decimal. The DCoffset value corresponds to an increase or a decrease expressed in unitof percent (%). For example, the DC offset value is set to a 20%increase. The DC offset generator 23 outputs the DC-offset-value signalto the color-difference-signal DC offset converter 21 and the secondsub-mixer 25.

The color-difference-signal DC offset converter 21 processes only “red”components of the color difference signals in the input contents datawhich represent pixels at the watermark-embedding addresses notified bythe embedding-position detector 22. In other words, thecolor-difference-signal DC offset converter 21 processes only “red”components of the input contents data which correspond to the prescribedblocks D representing the watermark, “JVC”. Specifically, thecolor-difference-signal DC offset converter 21 increases or decreasesthe DC values of the “red” signal components in accordance with the DCoffset value represented by the output signal from the DC offsetgenerator 23. For example, the DC values of the “red” signal componentsare increased by 20%. In this way, the “red” signal componentscorresponding to the prescribed blocks D are converted from theiroriginal states. On the other hand, the color-difference-signal DCoffset converter 21 does not process “blue” and “green” components ofthe color difference signals in the input contents data which representpixels in the watermark-embedding positions. Also, thecolor-difference-signal DC offset converter 21 does not process theluminance signal in the input contents data which represent pixels atthe watermark-embedding addresses. Furthermore, thecolor-difference-signal DC offset converter 21 does not process theinput contents data which represent pixels at addresses different fromthe watermark-embedding addresses. The color-difference-signal DC offsetconverter 21 feeds the processing-resultant contents data (theconversion-resultant contents data) to the pixel specified-bit detector27 and the second sub-mixer 25.

For every pixel represented by the luminance signal Y in theconversion-resultant contents data fed from the color-difference-signalDC offset converter 21, the pixel specified-bit detector 27 determinesthe number “N” of lower bits starting from the lowest bit among 8 bitscorresponding to the pixel in accordance with a desired strength(intensity) of an embedded watermark relative to an original picture.Here, “N” denotes a natural number smaller than 8. A fragment of awatermark can be embedded into the determined N lower bits. The pixelspecified-bit detector 27 receives a control signal representing thedesired strength of an embedded watermark. The control signal is fedfrom a suitable device such as an input device which can be accessed bya user.

When the desired strength of an embedded watermark relative to anoriginal picture is the smallest, the pixel specified-bit detector 27sets the number N to 1 and therefore designates and selects only thelowest bit among 8 bits per luminance pixel as a bit into which awatermark fragment can be embedded as shown in FIG. 5. When the desiredstrength of an embedded watermark relative to an original picture is thesecond smallest, the pixel specified-bit detector 27 sets the number Nto 2 and therefore designates and selects only the two lower bits among8 bits per luminance pixel as bits into which a watermark fragment canbe embedded as shown in FIG. 6. When the desired strength of an embeddedwatermark relative to an original picture is the third smallest, thepixel specified-bit detector 27 sets the number N to 3 and thereforedesignates and designates only the three lower bits among 8 bits perluminance pixel as bits into which a watermark fragment can be embeddedas shown in FIG. 7.

In this way, the number of N lower bits among 8 bits per luminance pixelinto which a watermark fragment can be embedded is increased as thedesired strength of the embedded watermark relative to an originalpicture is greater. In fact, the actual strength of an embeddedwatermark relative to an original picture rises in accordance with anincrease in the number of N lower bits among 8 bits per luminance pixelinto which a corresponding fragment of the watermark has been embedded.The original picture is less deteriorated as the strength of theembedded watermark decreases. In the case where watermark-added contentsdata are compressed at a high rate by an application program, thestrength of the embedded watermark is preferably set to such a greatvalue as to prevent the watermark from being erased by the compression.Thus, the strength of the embedded watermark is decided in accordancewith the application program.

Watermark information (information representing a watermark) has one bitper interested pixel. Accordingly, the watermark information is “0” or“1” for every interested pixel. The watermark information is alsoreferred to as the watermark data. In the case where only the lowest bitis selected among 8 bits per luminance pixel as a bit into which awatermark fragment can be embedded as shown in FIG. 5, a watermarkinformation bit of “0” or “1” corresponds to a lowest luminance pixelbit of “0” or “1”. In the case where only the two lower bits areselected among 8 bits per luminance pixel as bits into which a watermarkfragment can be embedded as shown in FIG. 6, a watermark information bitof “0” or “1” corresponds to two lower luminance pixel bits of “00” (“0”in decimal) or “11” (“3” in decimal). In the case where only the threelower bits are selected among 8 bits per luminance pixel as bits intowhich a watermark fragment can be embedded as shown in FIG. 7, awatermark information bit of “0” or “1” corresponds to three lowerluminance pixel bits of “000” (“0” in decimal) or “111” (“7” indecimal). As previously mentioned, the actual strength of an embeddedwatermark relative to an original picture rises in accordance with anincrease in the number of N lower bits among 8 bits per luminance pixelinto which a corresponding fragment of the watermark has been embedded.

The pixel specified-bit detector 27 outputs the designated luminancepixel bits (the selected luminance pixel bits) to the exclusive-ORdevice 29. The exclusive-OR device 29 includes a calculator whichcomputes a binary luminance pixel value from the designated luminancepixel bits. In the case where only the lowest bit is selected among 8bits per luminance pixel as a bit into which a watermark fragment can beembedded, the calculator sets a luminance pixel value to “0” for adesignated luminance pixel bit of “0”. The calculator sets a luminancepixel value to “1” for a designated luminance pixel bit of “1”. In thecase where only the two lower bits are selected among 8 bits perluminance pixel as bits into which a watermark fragment can be embedded,the calculator sets a luminance pixel value to “1” for two lowerluminance pixel bits of “11” (“3” in decimal). The calculator sets aluminance pixel value to “0” for two lower luminance pixel bitsdifferent from “11”. In the case where only the three lower bits areselected among 8 bits per luminance pixel as bits into which a watermarkfragment can be embedded, the calculator sets a luminance pixel value to“1” for three lower luminance pixel bits of “111” (“7” in decimal). Thecalculator sets a luminance pixel value to “0” for three lower luminancepixel bits different from “111”. The exclusive-OR device 29 receivesbits composing watermark information from the fixed-pattern generator28. The exclusive-OR device 29 processes the luminance pixel values bycalculation in response to the watermark information bits. Theexclusive-OR device 29 outputs the processing-resultant luminance pixelvalues to the second sub-mixer 25.

For each of the watermark-embedding addresses notified by theembedding-position detector 22, the fixed-pattern generator 28 produces,for example, 9 bits in a fixed pattern which correspond to 9 neighboringpixels being 3 luminance pixels in a horizontal direction by 3 luminancepixels in a vertical direction. FIG. 8 shows an example of the fixedpattern of 9 bits (3 by 3 bits). Preferably, the fixed-pattern generator28 includes a memory storing the fixed-pattern bits. The fixed-patterngenerator 28 feeds the fixed-pattern bits to the exclusive-OR device 29as watermark information bits. For other positions (addresses) in aframe, the fixed-pattern generator 28 does not feed the fixed-patternbits to the exclusive-OR device 29 or outputs a signal of “0” to theexclusive-OR device 29. The fixed-pattern generator 28 notifies theexclusive-OR device 29 of the watermark-embedding addresses.

The exclusive-OR device 29 includes a memory storing 9 bits in aspecified pattern which correspond to 9 neighboring pixels being 3luminance pixels in a horizontal direction by 3 luminance pixels in avertical direction. FIG. 9 shows an example of the specified pattern of9 bits (3 by 3 bits). The exclusive-OR device 29 also includes a sectionfor processing, in response to the watermark information bits, theluminance pixel values corresponding to the watermark-embeddingaddresses. Specifically, for each of the watermark-embedding addresses,the processing section of the exclusive-OR device 29 executesexclusive-OR operation between the watermark information bits (thefixed-pattern bits) and the specified-pattern bits on a bit-by-bitbasis. The processing section forcedly sets or equalizes thecorresponding luminance pixel values to the result of the exclusive-ORoperation, for example, “111111111”. The exclusive-OR device 29 outputsdata representative of the processing-resultant luminance pixel valuesto the second sub-mixer 25. The exclusive-OR device 29 may notify thesecond sub-mixer 25 of the exclusive-OR operation result and theaddresses of the related luminance pixel values.

In other words, the processing section of the exclusive-OR device 29calculates a 9-bit pattern which can be converted into the specified bitpattern (see FIG. 9) by exclusive-OR operation with the fixed bitpattern (see FIG. 8). The processing section forcedly sets or equalizesthe corresponding luminance pixel values to the calculated 9-bitpattern. An example of the calculated 9-bit pattern is “111111111”. Theexclusive-OR device 29 outputs data representative of theprocessing-resultant luminance pixel values (that is, the calculated9-bit pattern) to the second sub-mixer 25.

The second sub-mixer 25 places the output data from the exclusive-ORdevice 29 in the corresponding luminance pixel bits of theconversion-resultant contents data (the data outputted from thecolor-difference-signal DC offset converter 21) which are designated andselected by the pixel specified-bit detector 27. In other words, thesecond sub-mixer 25 replaces the contents of the designated luminancepixel bits (the selected luminance pixel bits) with the contents of theoutput data from the exclusive-OR device 29. Thus, the second sub-mixer25 rewrites or updates the designated luminance pixel bits intoagreement with the output data from the exclusive-OR device 29. As aresult, the calculated 9-bit pattern is embedded into each of portionsof the conversion-resultant contents data which correspond to thewatermark-embedding addresses. Basically, the second sub-mixer 25 doesnot change other bits of the conversion-resultant contents data. Thesecond sub-mixer 25 combines the updating-resultant designated luminancepixel bits and the other bits of the conversion-resultant contents datainto first watermarked contents data, that is, first watermark-addedcontents data.

The copyright information generator 14 produces a signal indicative of acopyright, for example, a copyright on the picture or picturesrepresented by the input contents data. The copyright informationgenerator 14 outputs the produced signal to the second sub-mixer 25 ascopyright information. The copyright information is composed of, forexample, about 32 bits of a binary code which indicate the ASCIIcharacters meaning the copyright or the ID number of a maker of thepicture or pictures represented by the input contents data.

The second sub-mixer 25 embeds the DC-offset-value signal and thecopyright information into the first watermark-added contents data in asuitable way selected from known ways. Information about the usedembedding way is possessed by not only the encoding side (the recordingside) but also a decoding side (a reproducing side). The embedding ofthe DC-offset-value signal and the copyright information converts thefirst watermark-added contents data into second watermark-added contentsdata (second watermarked contents data). The second sub-mixer 25 outputsthe second watermark-added contents data to the MPEG encoder 15.

The MPEG encoder 15 compressively encodes the second watermark-addedcontents data into watermark-added MPEG contents data according to aknown MPEG encoding procedure. The MPEG encoder 15 feeds thewatermark-added MPEG contents data to the data writer 16. The datawriter 16 changes the watermark-added MPEG contents data intocorresponding data of a prescribed format suited for being recorded on arecording medium 17. The data writer 16 records theprescribed-record-format contents data on the recording medium 17.

As understood from FIGS. 8 and 9, the fixed bit pattern produced by thefixed-pattern generator 28, the specified bit pattern, and the embeddedbit pattern remain unchanged when being rotated through 90, 180, or 270degrees. Accordingly, even in the case where the first or secondwatermark-added contents data are processed by steps including a step ofrotating the represented image through 90, 180, or 270 degrees, thewatermark can be detected from the processing-resultant contents data.

FIG. 10 shows an apparatus for reproducing or extracting a watermarkfrom watermarked contents data (watermark-added contents data) accordingto the first embodiment of this invention. The apparatus of FIG. 10includes a data reader 31, an MPEG decoder 32, a watermark detectingdevice 33, a parameter detecting device 34, a parameterinverse-converter 35, a display 36, a copyright information detector 37,and a memory 38. The data reader 31 is connected with the MPEG decoder32 and the watermark detecting device 33. The MPEG decoder 32 isconnected with the parameter inverse-converter 35. The watermarkdetecting device 33 is connected with the parameter detecting device 34and the copyright information detector 37. The parameter detectingdevice 34 is connected with the parameter inverse-converter 35. Theparameter inverse-converter 35 is connected with the display 36. Thecopyright information detector 37 is connected with the memory 38.

The watermark detecting device 33 has a first watermark detector 41 anda second watermark detector 42. The parameter detecting device 34 has awatermark map data detector 43 and a DC-offset-value detector 44. Theparameter inverse-converter 35 has a color-difference-signal DC offsetinverse-converter 45. The first watermark detector 41 is connected withthe data reader 31 and the watermark map data detector 43. The secondwatermark detector 42 is connected with the data reader 31, thecopyright information detector 37, and the DC-offset-value detector 44.The watermark map data detector 43 is connected with thecolor-difference-signal DC offset inverse-converter 45. TheDC-offset-value detector 44 is connected with thecolor-difference-signal DC offset inverse-converter 45. Thecolor-difference-signal DC offset inverse-converter 45 is connected withthe MPEG decoder 32 and the display 36.

As shown in FIG. 11, the first watermark detector 41 has a pixelspecified-bit detector 47, an exclusive-OR device 48, a fixed-patterngenerator 49. The pixel specified-bit detector 47 is connected with thedata reader 31 (see FIG. 10) and the exclusive-OR device 48. Theexclusive-OR device 48 is connected with the watermark map data detector43 (see FIG. 10) and the fixed-pattern generator 49.

The data reader 31 accesses a recording medium 17 loaded withwatermark-added contents data generated by, for example, the apparatusin FIG. 1. The data reader 31 reproduces the watermark-added contentsdata from the recording medium 17. The data reader 31 feeds thereproduced watermark-added contents data to the MPEG decoder 32, thepixel specified-bit detector 47, and the second watermark detector 42.The reproduced watermark-added contents data are composed of awatermark-added luminance signal Y and color difference signals Cb andCr. Each of the watermark-added luminance signal Y and the colordifference signals Cb and Cr has 8 bits per pixel.

The MPEG decoder 32 decodes the reproduced watermark-added contents dataaccording to an MPEG decoding procedure. The MPEG decoder 32 outputs thedecoding-resultant watermark-added contents data to thecolor-difference-signal DC offset inverse-converter 45.

For every pixel represented by the luminance signal Y in the reproducedwatermark-added contents data, the pixel specified-bit detector 47determines the number “N” of lower bits starting from the lowest bitamong 8 bits corresponding to the pixel in accordance with the strength(intensity) of the embedded watermark relative to the original picture.Here, “N” denotes a natural number smaller than 8. A fragment of thewatermark is embedded in the determined N lower bits. The pixelspecified-bit detector 47 receives a control signal representing thestrength of the embedded watermark. The control signal is fed from asuitable device such as an input device which can be accessed by a user.

Generally, the strength of the watermark embedded by the apparatus ofFIG. 1 is previously known to the apparatus of FIG. 10 since thewatermark strength is decided in accordance with the application programas previously mentioned. Preferably, a signal representing the strengthof the embedded watermark is not transmitted together with thewatermark-added contents data. Alternatively, a signal representing thestrength of the embedded watermark may be included in thewatermark-added contents data. According to a first example, theleft-upper corner pixel in every frame represented by thewatermark-added contents data is loaded with a signal representing thestrength of the embedded watermark. According to a second example, aheader of the watermark-added contents data is loaded with a signalrepresenting the strength of the embedded watermark. In these cases, thesignal representing the strength of the embedded watermark istransmitted together with the watermark-added contents data.

When the strength of the embedded watermark relative to the originalpicture is the smallest, the pixel specified-bit detector 47 sets thenumber N to 1 and therefore designates and selects only the lowest bitamong 8 bits per luminance pixel as a bit in which a fragment of thewatermark is embedded as shown in FIG. 5. When the strength of theembedded watermark relative to the original picture is the secondsmallest, the pixel specified-bit detector 47 sets the number N to 2 andtherefore designates and selects only the two lower bits among 8 bitsper luminance pixel as bits in which a fragment of the watermark isembedded as shown in FIG. 6. When the strength of the embedded watermarkrelative to the original picture is the third smallest, the pixelspecified-bit detector 47 sets the number N to 3 and thereforedesignates and designates only the three lower bits among 8 bits perluminance pixel as bits in which a fragment of the watermark is embeddedas shown in FIG. 7.

The pixel specified-bit detector 47 outputs the designated luminancepixel bits (the selected luminance pixel bits) to the exclusive-ORdevice 48. The exclusive-OR device 48 includes a calculator whichcomputes a binary luminance pixel value from the designated luminancepixel bits. In the case where only the lowest bit is selected among 8bits per luminance pixel as a bit in which a fragment of the watermarkis embedded, the calculator sets a luminance pixel value to “0” for adesignated luminance pixel bit of “0”. The calculator sets a luminancepixel value to “1” for a designated luminance pixel bit of “1”. In thecase where only the two lower bits are selected among 8 bits perluminance pixel as bits in which a fragment of the watermark isembedded, the calculator sets a luminance pixel value to “1” for twolower luminance pixel bits of “11” (“3” in decimal). The calculator setsa luminance pixel value to “0” for two lower luminance pixel bitsdifferent from “11”. In the case where only the three lower bits areselected among 8 bits per luminance pixel as bits in which a fragment ofthe watermark is embedded, the calculator sets a luminance pixel valueto “1” for three lower luminance pixel bits of “111” (“7” in decimal).The calculator sets a luminance pixel value to “0” for three lowerluminance pixel bits different from “111”.

The fixed-pattern generator 49 includes a memory storing bits in a fixedpattern. The number of the fixed-pattern bits is equal to, for example,9. The fixed bit pattern corresponds to, for example, 9 neighboringpixels being 3 luminance pixels in a horizontal direction by 3 luminancepixels in a vertical direction. The fixed bit pattern produced by thefixed-pattern generator 49 is the same as that produced by thefixed-pattern generator 28 in the apparatus of FIG. 1. For example, thefixed bit pattern produced by the fixed-pattern generator 49 isidentical with that in FIG. 8. The fixed-pattern generator 49 outputsthe bits in the fixed pattern to the exclusive-OR device 48.

For every 3 by 3 neighboring luminance pixels, the device 48 executesexclusive-OR operation between the luminance pixel values and thefixed-pattern bits on a bit-by-bit basis. The exclusive-OR device 48outputs data representative of the results of exclusive-OR operation tothe watermark map data detector 43.

The exclusive-OR device 48 includes a memory storing 9 bits in aspecified pattern which correspond to 9 neighboring pixels being 3luminance pixels in a horizontal direction by 3 luminance pixels in avertical direction. The specified bit pattern is the same as that usedin the apparatus of FIG. 1. For example, the specified bit pattern isidentical with that in FIG. 9. The exclusive-OR device 48 furtherincludes a deciding section (a comparing section). For every 3 by 3neighboring luminance pixels, the deciding section determines whether ornot the bit pattern represented by the operation-result data is equal tothe specified bit pattern. When the bit pattern represented by theoperation-result data is equal to the specified bit pattern, thedeciding section determines that the operation-resultant data have awatermark fragment. In this case, the deciding section computes theposition of 3 by 3 neighboring luminance pixels represented by theoperation-resultant data. The computed position is defined relative tothe frame. The computed position is referred as the watermark position.The deciding section notifies the watermark position to the watermarkmap data detector 43. On the other hand, when the bit patternrepresented by the operation-result data is different from the specifiedbit pattern, the deciding section determines that theoperation-resultant data do not have any watermark fragment. In thiscase, the deciding section does not implement the position computingstep.

The watermark map data detector 43 generates a central address of arectangular block of, for example, 9 neighboring luminance pixels (3luminance pixels in a horizontal direction by 3 luminance pixels in avertical direction) from each of the watermark positions notified by theexclusive-OR device 48. The rectangular blocks correspond to theprescribed blocks D representing the watermark, for example, “JVC” inFIG. 4. The rectangular blocks D are also referred to as the watermarkblocks D. The watermark map data detector 43 notifies the generatedwatermark block addresses to the color-difference-signal DC offsetinverse-converter 45.

The watermark map data detector 43 may include a memory storing fontinformation representing the thicknesses and shapes of letters andcharacters. In this case, a reproduced watermark can be indicated ashigh-quality letters and characters on the display 36.

The second watermark detector 42 detects a DC-offset-value signal andcopyright information in the reproduced watermark-added contents data.The second watermark detector 42 extracts the DC-offset-value signal andthe copyright information from the reproduced watermark-added contentsdata. The second watermark detector 42 feeds the DC-offset-value signalto the DC-offset-value detector 44. The second watermark detector 42feeds the copyright information to the copyright information detector37.

The copyright information detector 37 decodes the copyright information.The copyright information detector 37 stores the decoding-resultantcopyright information into the memory 38.

The DC-offset-value detector 44 recovers a DC offset value representedby the DC-offset-value signal. The DC-offset-value detector 44 notifiesthe recovered DC offset value to the color-difference-signal DC offsetinverse-converter 45.

The color-difference-signal DC offset inverse-converter 45 receives thewatermark-added contents data from the MPEG decoder 32. Thecolor-difference-signal DC offset inverse-converter 45 processes only“red” components of the color difference signals in the watermark-addedcontents data which represent pixels in the watermark blocks having theaddresses notified by the watermark map data detector 43. In otherwords, the color-difference-signal DC offset inverse-converter 45processes only “red” components of the watermark-added contents datawhich correspond to the prescribed blocks D representing the watermark,“JVC”. Specifically, the color-difference-signal DC offsetinverse-converter 45 carries out DC-offset-value conversion inverse withrespect to that implemented by the color-difference-signal DC offsetconverter 21 in the apparatus of FIG. 1. In more detail, thecolor-difference-signal DC offset inverse-converter 45 increases ordecreases the DC values of the “red” signal components in accordancewith the DC offset value notified by the DC-offset-value detector 44.For example, the DC values of the “red” signal components are decreasedby about 17% to cancel the increases provided by thecolor-difference-signal DC offset converter 21 in the apparatus ofFIG. 1. In this way, the “red” signal components corresponding to theprescribed blocks D are inversely converted into their original states.On the other hand, the color-difference-signal DC offsetinverse-converter 45 does not process “blue” and “green” components ofthe color difference signals in the watermark-added contents data whichrepresent pixels in the watermark blocks. The color-difference-signal DCoffset inverse-converter 45 passes these unprocessed “blue” and “green”components of the color difference signals to the display 36. Also, thecolor-difference-signal DC offset inverse-converter 45 does not processthe luminance signal in the watermark-added contents data whichrepresent pixels in the watermark blocks. The color-difference-signal DCoffset inverse-converter 45 passes the unprocessed luminance signal tothe display 36. Furthermore, the color-difference-signal DC offsetinverse-converter 45 does not process the watermark-added contents datawhich represent pixels outside the watermark blocks. Thecolor-difference-signal DC offset inverse-converter 45 outputs theunprocessed watermark-added contents data to the display 36.

The display 36 indicates an image represented by the watermark-addedcontents data outputted from the color-difference-signal DC offsetinverse-converter 45. Since the “red” components of the watermark-addedcontents data which correspond to the watermark blocks D have beenreturned to their original states by the color-difference-signal DCoffset inverse-converter 45, the indicated image is identical withoriginal one free from the watermark, “JVC”.

In the event that the DC offset value fails to be recovered, the “red”components of the watermark-added contents data which correspond to thewatermark blocks D can not be returned to their original states by thecolor-difference-signal DC offset inverse-converter 45. Thus, in thiscase, the “red” signal components having the 20%-increased DC values arepassed to the display 36 as they are. As a result, in the imageindicated by the display 36, pixels in the watermark blocks D areforcedly colored red while other pixels have original colors. Therefore,the watermark (“JVC”) is displayed as red portions of the indicatedimage which form the foreground in the original picture.

In the case where illegal conduct intended to remove the watermark fromthe watermark-added contents data deletes the DC-offset-value signaltherefrom, the DC offset value can not be recovered from the resultantcontents data since the DC-offset-value signal is absent therefrom.Thus, in this case, when the resultant contents data are indicated by adisplay, the indicated image has a conspicuous red watermark whichdeteriorates the image quality. Accordingly, it is possible to protectthe copyrighted contents data.

Second Embodiment

FIG. 12 shows an apparatus for embedding or inserting a watermark intocontents data according to a second embodiment of this invention. Theapparatus of FIG. 12 is similar to the apparatus of FIG. 1 except fordesign changes mentioned hereafter.

The apparatus of FIG. 12 includes a transmitter 51 connected between theMPEG encoder 15 and a transmission path. The transmitter 51 receives thewatermark-added MPEG contents data from the MPEG encoder 15. Thetransmitter 51 outputs the watermark-added MPEG contents data to thetransmission path. The watermark-added MPEG contents data propagatealong the transmission path. Preferably, the transmitter 51 implementsformat conversion of the watermark-added MPEG contents data beforeoutputting the resultant data to the transmission path.

The transmission path includes a communication network such as a wirecommunication network, a radio communication network, or the Internet.

FIG. 13 shows an apparatus for reproducing or extracting a watermarkfrom watermarked contents data (watermark-added contents data) accordingto the second embodiment of this invention. The apparatus of FIG. 13 issimilar to the apparatus of FIG. 10 except for design changes mentionedhereafter.

The apparatus of FIG. 13 includes a receiver 53 connected to atransmission path, the MPEG decoder 32, and the watermark detectingdevice 33. The device 53 receives watermark-added contents data from thetransmission path. Originally, the watermark-added contents data aretransmitted by the apparatus of FIG. 12. The receiver 53 feeds thereceived watermark-added contents data to the MPEG decoder 32 and thewatermark detecting device 33. Preferably, the receiver 53 implementsformat conversion of the received watermark-added contents data beforeoutputting the resultant data to the MPEG decoder 32 and the watermarkdetecting device 33.

Third Embodiment

A third embodiment of this invention is similar to the first embodimentor the second embodiment thereof except for design changes mentionedhereafter.

In the third embodiment of this invention, the color-difference-signalDC offset converter 21 (see FIG. 1) processes only specified-colorcomponents of the color difference signals in the input contents datawhich represent pixels at the watermark-embedding addresses notified bythe embedding-position detector 22. The specified color differs from“red”.

In the third embodiment of this invention, the color-difference-signalDC offset inverse-converter 45 (see FIG. 10) processes onlyspecified-color components of the color difference signals in thewatermark-added contents data which represent pixels in the watermarkblocks having the addresses notified by the watermark map data detector43. The specified color differs from “red”.

Fourth Embodiment

A fourth embodiment of this invention is similar to the first embodimentor the second embodiment thereof except for design changes mentionedhereafter.

In the fourth embodiment of this invention, the parameter converter 11(see FIG. 1) processes only segments of the luminance signal in theinput contents data which represent pixels at the watermark-embeddingaddresses.

In the fourth embodiment of this invention, the parameterinverse-converter 35 (see FIG. 10) processes only segments of theluminance signal in the watermark-added contents data which representpixels in the watermark blocks.

Fifth Embodiment

A fifth embodiment of this invention is similar to the first embodimentor the second embodiment thereof except for design changes mentionedhereafter. The fifth embodiment of this invention implements parameterconversion such as DC-offset-value conversion, color-space matrixconversion, or data-sample exchange conversion.

The DC-offset-value conversion is designed so that Cb data or Cr dataare reduced by 20%. In the case where the conversion-resultant contentsdata are indicated on a display without being subjected to inverseconversion, the whole of the indicated image is tinged with red or blue.Thus, in this case, the quality of the indicated image is low.

Two parameters may be set in Cb data or Cr data. Conversion isimplemented about the two parameters. In this case, the parameterconversion is applied to both two color spaces (blue and red spaces).The parameter conversion may be based on a prescribed function such as“Y=a·X+b”, where “X” denotes original Cb or Cr data, and “a” and “b”denote parameters. In this case, for one value, an 8-bit informationpiece is embedded as a fragment of a watermark.

A luminance signal Y, a color difference signal Cb, and a colordifference signal Cr define three color spaces to be compressivelyencoded. The three color spaces may be converted into other color spacesby prescribed calculative operation with respect to matrices each having3 rows and 3 columns. In this case, 9 parameters are used andtransmitted as watermark information. The 9 parameters are 9 matrixelements (9 matrix coefficients) which may be represented by 9 bits.Alternatively, 8 different types of matrices may be predetermined. Inthis case, a 3-bit signal representing the type of each matrix istransmitted.

Data samples may be converted on a pixel-by-pixel basis. For example,regarding a sequence of 1-pixel-corresponding Cb and Cr data pieces in araster scanning order, Cb and Cr data pieces corresponding to each ofeven-numbered pixels are exchanged. In a same color space, odd-numbereddata pieces and even-numbered data pieces may be exchanged. Preferably,a parameter which can uniquely designate rules of the exchange isdecided in both an encoding side and a decoding side (a recording sideand a reproducing side). In this case, information about the parameteris transmitted, and inverse conversion is executed in response to thetransmitted parameter. When the information about the parameter fails tobe transmitted, the quality of an indicated image (a reproduced image)is low.

In the case of contents data representing a picture or pictures,parameter conversion may be designed so that figures and charactershaving a given visual meaning will be included in an image representedby reproduced data occurring before inverse conversion. In this case,DC-value-offset conversion or color-space matrix conversion in theinverse conversion results in erasure of the figures and characters.Preferably, information about the positions of the figures andcharacters relative to a frame is transmitted in such a manner that auser can hardly perform detection thereof. For example, the informationabout the positions of the figures and characters is transmitted as awatermark pattern.

Sixth Embodiment

FIG. 14 shows an apparatus for embedding or inserting a watermark intocontents data according to a sixth embodiment of this invention. Theapparatus of FIG. 14 includes a watermark mixer 111, a copyrightinformation generator 112, a jump-destination address generator 113, anMPEG encoder 114, a data rearranging device 115, and a data writer 116.

The watermark mixer 111 is connected with the copyright informationgenerator 112, the jump-destination address generator 113, and the MPEGencoder 114. The jump-destination address generator 113 is connectedwith the data rearranging device 115. The MPEG encoder 114 is connectedwith the data rearranging device 115. The data rearranging device 115 isconnected with the data writer 116.

Input contents data representing an original picture or a sequence oforiginal pictures are fed to the watermark mixer 111. The input contentsdata may additionally include audio information.

The copyright information generator 112 produces a signal indicative ofa copyright, for example, a copyright on the picture or picturesrepresented by the input contents data. The copyright informationgenerator 112 feeds the produced signal to the watermark mixer 111 ascopyright information. The copyright information is composed of, forexample, about 32 bits of a binary code which indicate the ASCIIcharacters meaning the copyright or the ID number of a maker of thepicture or pictures represented by the input contents data.

MPEG-encoded contents data are divided into unit segments called“cells”. The jump-destination address generator 113 produces ajump-destination address for each cell. According to the producedjump-destination addresses, cells are rearranged in an order differentfrom an original playback order (a normal playback order). Thejump-destination address generator 113 converts every jump-designationaddress into a 3-bit code word, and feeds the 3-bit code word to thewatermark mixer 111 and the data rearranging device 115.

The copyright information and the jump-destination address (the 3-bitcode word) fed to the watermark mixer 111 compose watermark information.The watermark mixer 111 embeds or inserts the watermark information intothe input contents data to get watermarked contents data orwatermark-added contents data. The watermark mixer 111 outputs thewatermark-added contents data to the MPEG encoder 114.

The MPEG encoder 114 compressively encodes the watermark-added contentsdata into watermark-added MPEG contents data according to a known MPEGencoding procedure. The MPEG encoder 114 feeds the watermark-added MPEGcontents data to the data rearranging device 115.

As previously mentioned, the jump-destination address generator 113notifies the data rearranging device 115 of a jump-designation addressfor each cell of the watermark-added MPEG contents data. The datarearranging device 115 rearranges the cells of the watermark-added MPEGcontents data in an order being different from the original playbackorder and being determined by the jump-destination addresses. The datarearranging device 115 outputs the cells of the watermark-added MPEGcontents data to the data writer 116 in the rearranging-resultant order.Thus, the data rearranging device 115 converts the watermark-added MPEGcontents data into second watermark-added MPEG contents data. The datarearranging device 115 outputs the second watermark-added MPEG contentsdata to the data writer 116. The data writer 116 changes the secondwatermark-added MPEG contents data into corresponding data of aprescribed format suited for being recorded on a recording medium 117.The data writer 116 records the prescribed-record-format contents dataon the recording medium 117.

The sixth embodiment of this invention will be further described below.Pieces of contents data are rearranged in an order different from anoriginal playback order on a unit-by-unit basis (for example, acell-by-cell basis) before being recorded on a recording medium.Jump-destination addresses relating to the data-piece rearrangement areembedded or inserted into the contents data as fragments of watermarkinformation. For example, each cell is additionally provided with ajump-destination address (a 3-bit code word) representing the positionof the head of a next cell which should follow the present cell in theoriginal playback order. During playback, jump-destination addresses arereproduced, and jumps between cells are executed in response to thereproduced jump-destination addresses to provide a cell sequenceaccorded with the original playback order.

A recording medium stores contents data including video informationwhich results from MPEG-based data compression. According to theMPEG-based data compression, a motion compensator generates datarepresenting a decoding-resultant picture. An error between datarepresenting an input picture and the data representing thedecoding-resultant picture is calculated. Therefore,temporally-redundant portions of the input-picture data are reduced ordeleted. The MPEG-based data compression implements prediction in adirection from the past, prediction in a direction from the future, orprediction in both a direction from the past and a direction from thefuture. The implemented prediction is changed on an MB-by-MB basis,where MB denotes a macro-block of picture data which corresponds to 16pixels by 16 pixels. The direction of the implemented prediction isdecided by the type of an input picture. Encoding data representative ofa P picture is changed between first and second modes. According to thefirst mode, macro-blocks of the P-picture data are encoded by using theprediction in the direction from the past. According to the second mode,macro-blocks of the P-picture data are independently encoded withoutusing any prediction. Encoding data representative of a B picture ischanged among first, second, third, and fourth modes. According to thefirst mode, macro-blocks of the B-picture data are encoded by using theprediction in the direction from the future. According to the secondmode, macro-blocks of the B-picture data are encoded by using theprediction in the direction from the past. According to the third mode,macro-blocks of the B-picture data are encoded by using the predictionin both the direction from the future and the direction from the past.According to the fourth mode, macro-blocks of the B-picture data areindependently encoded without using any prediction. Macro-blocks of datarepresentative of an I picture are independently encoded without usingany prediction.

Motion compensation implements pattern matching between two successivepictures on an MB-by-MB basis, thereby detecting motion vectors havingan accuracy corresponding to a half pel (a half pixel). Shifts ofpicture segments are executed, and prediction is implemented on thebasis of the shift-resultant picture segments. Motion vectors are in ahorizontal direction and a vertical direction. There is an MC-modesignal (a motion-compensation-mode signal) indicating the startingposition for the prediction. Motion vectors and an MC-mode signal aretransmitted as added information related to every macro-block.Successive pictures starting from an I picture to a pictureimmediately-preceding a next I picture are referred to as a group ofpictures (GOP). In general, one GOP is composed of about 15 pictures.Compression-resultant data are divided into unit portions of a commonlogical structure in which video data representative of one GOP andaudio data corresponding to the playback time of the video data aremultiplexed with each other. A unit portion of the compression-resultantdata is referred to as a cell.

Cells are reproduced in an original playback order as shown in FIG. 15so that a program represented by the cells is continuously played back.Cells are rearranged in an order different from the original playbackorder before being recorded. As a result, the program represented by thecells is scrambled. FIG. 16 shows a first example of therearranging-resultant order which is caused by jumping some cells toprevious positions or later positions. FIG. 17 shows a second example ofthe rearranging-resultant order which is caused by jumping some cells toprevious positions or later positions.

A jump-destination address is represented by a word of a 3-bit code foreach cell. Every 3-bit code word indicates the type of a jump by which arelated cell can be returned to its correct position in an originalplayback order. Specifically, a jump-destination-address code word of“000” indicates that a jump is unnecessary. A jump-destination-addresscode word of “001” indicates a jump over one cell in the forwarddirection. A jump-destination-address code word of “010” indicates ajump over two cells in the forward direction. A jump-destination-addresscode word of “011” indicates a jump over three cells in the forwarddirection. A jump-destination-address code word of “100” indicates ajump over one cell in the backward direction. A jump-destination-addresscode word of “101” indicates a jump over two cells in the backwarddirection. A jump-destination-address code word of “110” indicates ajump over three cells in the backward direction. Ajump-destination-address code word of “111” indicates a jump over fourcells in the backward direction.

In the case where a jump-destination-address code word of “000” isdetected during playback, a jump from a related cell is inhibited. Whena jump-destination-address code word of “001” is detected, a jump from arelated cell over one cell in the forward direction is executed. When ajump-destination-address code word of “100” is detected, a jump from arelated cell over one cell in the backward direction is executed. When ajump-destination-address code word of “111” is detected, a jump from arelated cell over four cells in the backward direction is executed.

Regarding the rearranging-resultant order in FIG. 16, ajump-destination-address code word of “001” is added to the end of thefirst cell (the leftmost cell). A jump-destination-address code word of“101” is added to the end of the second cell.

Regarding the rearranging-resultant order in FIG. 17, ajump-destination-address code word of “010” is added to the end of thefirst cell (the leftmost cell). A jump-destination-address code word of“110” is added to the end of the second cell.

As previously mentioned, the jump-destination-address code words areembedded or inserted into the cells of the contents data as thefragments of the watermark information respectively.

FIG. 18 shows an apparatus for reproducing or extracting a watermarkfrom watermarked contents data (watermark-added contents data) accordingto the sixth embodiment of this invention. The apparatus of FIG. 18includes a data reader 121, an MPEG decoder 122, a watermark detectingdevice 123, a display 124, a jump-destination address calculator 125, acopyright information detector 126, a memory 127, and a loudspeaker 128.

The data reader 121 is connected with the MPEG decoder 122, thewatermark detecting device 123, and the jump-destination addresscalculator 125. The MPEG decoder 122 is connected with the display 124and the loudspeaker 128. The watermark detecting device 123 is connectedwith the jump-destination address calculator 125 and the copyrightinformation detector 126. The copyright information detector 126 isconnected with the memory 127.

A recording medium 117 stores watermark-added contents data originallygenerated by, for example, the apparatus of FIG. 14. The storedwatermark-added contents data have cells arranged in an order differentfrom an original playback order. The stored watermark-added contentsdata include a set of audio information and video information placed inthe cells. The stored watermark-added contents data also include ajump-destination-address code word for every cell which indicates thetype of a jump by which the related cell can be returned to its correctposition in the original playback order. The stored watermark-addedcontents data further include copyright information. Thejump-destination-address code words and the copyright informationcompose watermark information.

At an initial stage of operation of the apparatus in FIG. 18, the datareader 121 reads out the first cell of the watermark-added contents datafrom the recording medium 117. The data reader 121 feeds the first cellof the watermark-added contents data to the MPEG decoder 122 and thewatermark detecting device 123. The MPEG decoder 122 subjects the firstcell of the watermark-added contents data to an MPEG decoding procedure,thereby reproducing a video signal and an audio signal. The MPEG decoder122 feeds the reproduced video signal to the display 124. The reproducedvideo signal is indicated on the display 124. Preferably, the watermarkinformation is designed to be substantially invisible when beingindicated on the display 124. The MPEG decoder 122 feeds the reproducedaudio signal to the loudspeaker 128. The reproduced audio signal isconverted into corresponding sounds by the loudspeaker 128. On the otherhand, the watermark detecting device 123 extracts the watermarkinformation from the first cell of the watermark-added contents data.The extracted watermark information includes thejump-destination-address code word relating to the first cell. Thewatermark detecting device 123 feeds the jump-destination-address codeword to the jump-destination address calculator 125. Thejump-destination address calculator 125 computes a jump-destinationaddress from the jump-destination-address code word. Thejump-designation address denotes the position to which thecurrently-accessed point on the recording medium 117 should be jumped,that is, the position of the head of a cell to be accessed next. Thejump-destination address calculator 125 notifies the data reader 121 ofthe computed jump-destination address. After the read-out of the firstcell of the watermark-added contents data from the recording medium 117has been completed, the data reader 121 jumps the currently-accessedpoint on the recording medium 117 to the position denoted by thejump-destination address and then starts reading out the second cell ofthe watermark-added contents data from the recording medium 117.

Similarly, the data reader 121 reads out the second and later cells ofthe watermark-added contents data from the recording medium 117. Thedata reader 121 feeds the second and later cells of the watermark-addedcontents data to the MPEG decoder 122 and the watermark detecting device123. The MPEG decoder 122 subjects the second and later cells of thewatermark-added contents data to an MPEG decoding procedure, therebyreproducing a video signal and an audio signal. The MPEG decoder 122feeds the reproduced video signal to the display 124. The reproducedvideo signal is indicated on the display 124. The MPEG decoder 122 feedsthe reproduced audio signal to the loudspeaker 128. The reproduced audiosignal is converted into corresponding sounds by the loudspeaker 128. Onthe other hand, the watermark detecting device 123 extracts thewatermark information from the second and later cells of thewatermark-added contents data. The extracted watermark informationincludes the jump-destination-address code words relating to the secondand later cells. The watermark detecting device 123 feeds thejump-destination-address code words to the jump-destination addresscalculator 125. The jump-destination address calculator 125 computes ajump-destination address from each of the jump-destination-address codewords. The jump-designation address denotes the position to which thecurrently-accessed point on the recording medium 117 should be jumped,that is, the position of the head of a cell to be accessed next. Thejump-destination address calculator 125 notifies the data reader 121 ofthe computed jump-destination addresses. After the read-out of thecurrent cell of the watermark-added contents data from the recordingmedium 117 has been completed, the data reader 121 jumps thecurrently-accessed point on the recording medium 117 to the positiondenoted by the jump-destination address and then starts reading out thenext cell of the watermark-added contents data from the recording medium117.

As understood from the previous description, the currently-accessedpoint on the recording medium 117 is jumped or non-jumped in response toeach of the jump-destination-address code words. The jumps of thecurrently-accessed point in response to the jump-destination-addresscode words enable the cells of the watermark-added contents data to besequentially reproduced in the original playback order.

The watermark information extracted by the watermark detecting device123 includes the copyright information. The watermark detecting device123 feeds the copyright information to the copyright informationdetector 126. The copyright information detector 126 decodes thecopyright information. The copyright information detector 126 stores thedecoding-resultant copyright information into the memory 127. Aspreviously mentioned, the watermark information extracted by thewatermark detecting device 123 includes the jump-destination-addresscode words. The watermark detecting device 123 may store thejump-destination-address code words into the memory 127.

As understood from the previous description, when thejump-destination-address code words fail to be recovered, it isdifficult to reproduce the cells of the watermark-added contents data inthe normal playback order (the original playback order). In the eventthat illegal conduct removes the watermark information from thewatermark-added contents data, the jump-destination-address code wordsin the watermark information are also lost. Thus, in this case, thecells of the watermark-added contents data can not be reproduced in thenormal playback order. Accordingly, the watermark-added contents dataare protected from illegal playback.

Seventh Embodiment

A seventh embodiment of this invention is similar to the sixthembodiment thereof except for design changes mentioned hereafter. In theseventh embodiment of this invention, every frame (for example, everystill-picture frame) represented by watermark-added contents data isdivided into 4-by-4 sub-regions, that is, 16 sub-regions as shown inFIG. 19. For every frame, the watermark-added contents data are dividedinto unit segments called “blocks” corresponding to the 16 sub-regionsrespectively. For every frame, the blocks of the watermark-addedcontents data are arranged in a raster scanning order as shown in FIG.19.

In the seventh embodiment of this invention, the jump-destinationaddress generator 113 (see FIG. 14) produces a jump-destination addressfor each block. According to the produced jump-destination addresses,blocks are rearranged in an order different from the raster scanningorder. The jump-destination address generator 113 converts everyjump-designation address into a 3-bit code word, and feeds the 3-bitcode word to the watermark mixer 111 and the data rearranging device 115(see FIG. 14).

The copyright information and the jump-destination address (the 3-bitcode word) fed to the watermark mixer 111 compose watermark information.The watermark mixer 111 embeds or inserts the watermark information intothe input contents data to get watermarked contents data, that is,watermark-added contents data. The watermark mixer 111 outputs thewatermark-added contents data to the MPEG encoder 114 (see FIG. 14). Thewatermark mixer 111 may directly output the watermark-added contentsdata to the data rearranging device 115. In this case, the MPEG encoder114 is omitted.

The MPEG encoder 114 compressively encodes the watermark-added contentsdata into watermark-added MPEG contents data according to a known MPEGencoding procedure. The MPEG encoder 114 outputs the watermark-addedMPEG contents data to the data rearranging device 115 (see FIG. 14).

The jump-destination address generator 113 notifies the data rearrangingdevice 115 of a jump-designation address for each block of thewatermark-added contents data outputted from the watermark mixer 111 orthe MPEG encoder 114. The data rearranging device 115 rearranges theblocks of the watermark-added contents data from the watermark mixer 111or the MPEG encoder 114 in an order being different from the rasterscanning order and being determined by the jump-destination addresses.The data rearranging device 115 outputs the blocks of thewatermark-added contents data to the data writer 116 (see FIG. 14) inthe rearranging-resultant order. Thus, the data rearranging device 115converts the watermark-added contents data into second watermark-addedcontents data. The data rearranging device 115 outputs the secondwatermark-added contents data to the data writer 116. The data writer116 changes the second watermark-added contents data into correspondingdata of a prescribed format suited for being recorded on a recordingmedium 117 (see FIG. 17). The data writer 116 records theprescribed-record-format contents data on the recording medium 117.

The seventh embodiment of this invention will be further describedbelow. Pieces of contents data for every frame are rearranged in anorder different from a raster scanning order on a unit-by-unit basis(for example, a block-by-block basis) before being recorded on arecording medium. Jump-destination addresses relating to the data-piecerearrangement are embedded or inserted into the contents data asfragments of watermark information. For example, each block isadditionally provided with a jump-destination address (a 3-bit codeword) representing the position of the head of a next block which shouldfollow the present block in the raster scanning order. During playback,jump-destination addresses are reproduced, and jumps between blocks areexecuted in response to the reproduced jump-destination addresses toprovide a block sequence accorded with the raster scanning order.

Blocks are reproduced in the raster scanning order as shown in FIG. 19so that every 1-frame picture represented by the blocks is normallyreproduced. Blocks are rearranged in an order different from the rasterscanning order before being recorded. As a result, the 1-frame picturerepresented by the blocks is scrambled. FIG. 20 shows an example of therearranging-resultant order which is caused by jumping some blocks toprevious positions or later positions for a frame. The jump-based blockrearrangement keeps prescribed rules as follows. For every frame, blockscorresponding to two upper lines of sub-regions compose a first slice,and also blocks corresponding to two lower lines of sub-regions composea second slice which follows the first slice in the raster scanningorder. Slices are arranged in a fixed order. Any block jump betweendifferent slices is inhibited. Only block jumps in a same slice arepermitted. The first block in every slice remains corresponding to theleftmost upper sub-region. Accordingly, the last block in every slice isfollowed by the block in a next slice which corresponds to the leftmostupper sub-region.

A jump-destination address is represented by a word of a 3-bit code foreach block. Every 3-bit code word indicates the type of a jump by whicha related block can be returned to its correct position in the rasterscanning order. Specifically, a jump-destination-address code word of“000” indicates a jump by one block in the rightward direction plus ajump by one block in the upward direction. Here, the directions aredefined with respect to corresponding sub-regions in a frame. Ajump-destination-address code word of “001” indicates a jump by oneblock in the rightward direction. A jump-destination-address code wordof “010” indicates a jump by one block in the rightward direction plus ajump by one block in the downward direction. A jump-destination-addresscode word of “011” indicates a jump by one block in the upwarddirection. A jump-destination-address code word of “100” indicates ajump by one block in the downward direction. A jump-destination-addresscode word of “101” indicates a jump by one block in the leftwarddirection plus a jump by one block in the downward direction. Ajump-destination-address code word of “110” indicates a jump by oneblock in the leftward direction. A jump-destination-address code word of“111” indicates a jump by one block in the leftward direction plus ajump by one block in the upward direction.

Regarding the rearranging-resultant order in FIG. 20, for every frame, ajump-destination-address code word of “010” is added to the end of thefirst block. A jump-destination-address code word of “110” is added tothe end of the second block. A jump-destination-address code word of“000” is added to the end of the third block. A jump-destination-addresscode word of “001” is added to the end of the fourth block. Ajump-destination-address code word of “100” is added to the end of thefifth block. A jump-destination-address code word of “001” is added tothe end of the sixth block. A jump-destination-address code word of“001” is added to the end of the seventh block. Nojump-destination-address code word is added to the end of the eighthblock. The first to eighth blocks compose a slice. A next slice startsfrom the ninth block. A jump-destination-address code word of “001” isadded to the end of the ninth block. A jump-destination-address codeword of “101” is added to the end of the tenth block. Ajump-destination-address code word of “001” is added to the end of theeleventh block. A jump-destination-address code word of “000” is addedto the end of the twelfth block. A jump-destination-address code word of“100” is added to the end of the thirteenth block. Ajump-destination-address code word of “000” is added to the end of thefourteenth block. A jump-destination-address code word of “100” is addedto the end of the fifteenth block. No jump-destination-address code wordis added to the end of the sixteenth block.

In the seventh embodiment of this invention, watermark-added contentsdata stored in a recording medium 117 (see FIG. 18) have blocks arrangedin an order different from a raster scanning order. The storedwatermark-added contents data include a set of audio information andvideo information placed in the blocks. Alternatively, the storedwatermark-added contents data may include video information (forexample, still-picture information) placed in the blocks. The storedwatermark-added contents data also include a jump-destination-addresscode word for every block which indicates the type of a jump by whichthe related block can be returned to its correct position in the rasterscanning order. The stored watermark-added contents data further includecopyright information. The jump-destination-address code words and thecopyright information compose watermark information.

At an initial stage of 1-frame reproduction of the watermark-addedcontents data from the recording medium 117, the data reader 121 (seeFIG. 18) reads out the first block of the watermark-added contents datafrom the recording medium 117. The data reader 121 feeds the first blockof the watermark-added contents data to the MPEG decoder 122 and thewatermark detecting device 123 (see FIG. 18). The data reader 121 maydirectly feed the first block of the watermark-added contents data tothe display 124 (see FIG. 18). In this case, the MPEG decoder 122 isomitted. The watermark detecting device 123 extracts the watermarkinformation from the first block of the watermark-added contents data.The extracted watermark information includes thejump-destination-address code word relating to the first block. Thewatermark detecting device 123 feeds the jump-destination-address codeword to the jump-destination address calculator 125 (see FIG. 18). Thejump-destination address calculator 125 computes a jump-destinationaddress from the jump-destination-address code word. Thejump-designation address denotes the position to which thecurrently-accessed point on the recording medium 117 should be jumped,that is, the position of the head of a block to be accessed next. Thejump-destination address calculator 125 notifies the data reader 121 ofthe computed jump-destination address. After the read-out of the firstblock of the watermark-added contents data from the recording medium 117has been completed, the data reader 121 jumps the currently-accessedpoint on the recording medium 117 to the position denoted by thejump-destination address and then starts reading out the second block ofthe watermark-added contents data from the recording medium 117.

Similarly, the data reader 121 reads out the second and later blocks ofthe watermark-added contents data from the recording medium 117. Thedata reader 121 feeds the second and later blocks of the watermark-addedcontents data to the MPEG decoder 122 and the watermark detecting device123. In the absence of the MPEG decoder 122, the data reader 121directly feeds the second and later blocks of the watermark-addedcontents data to the display 124. The watermark detecting device 123extracts the watermark information from the second and later blocks ofthe watermark-added contents data. The extracted watermark informationincludes the jump-destination-address code words relating to the secondand later blocks. The watermark detecting device 123 feeds thejump-destination-address code words to the jump-destination addresscalculator 125. The jump-destination address calculator 125 computes ajump-destination address from each of the jump-destination-address codewords. The jump-designation address denotes the position to which thecurrently-accessed point on the recording medium 117 should be jumped,that is, the position of the head of a block to be accessed next. Thejump-destination address calculator 125 notifies the data reader 121 ofthe computed jump-destination addresses. After the read-out of thecurrent block of the watermark-added contents data from the recordingmedium 117 has been completed, the data reader 121 jumps thecurrently-accessed point on the recording medium 117 to the positiondenoted by the jump-destination address and then starts reading out thenext block of the watermark-added contents data from the recordingmedium 117.

The MPEG decoder 122 receives the blocks of the watermark-added contentsdata from the data reader 121. The MPEG decoder 122 subjects thewatermark-added contents data to an MPEG decoding procedure, therebyreproducing a video signal and an audio signal. The MPEG decoder 122feeds the reproduced video signal to the display 124 (see FIG. 18). Thereproduced video signal is indicated on the display 124. The MPEGdecoder 122 feeds the reproduced audio signal to the loudspeaker 128(see FIG. 18). The reproduced audio signal is converted intocorresponding sounds by the loudspeaker 128.

In the absence of the MPEG decoder 122, the display 124 directlyreceives the blocks of the watermark-added contents data from the datareader 121. The display 124 indicates a picture or pictures (forexample, a still picture or still pictures) represented by the blocks ofthe watermark-added contents data.

As understood from the previous description, the currently-accessedpoint on the recording medium 117 is jumped in response to each of thejump-destination-address code words. The jumps of the currently-accessedpoint in response to the jump-destination-address code words enable theblocks of the watermark-added contents data to be sequentiallyreproduced in the raster scanning order.

When the jump-destination-address code words fail to be recovered, it isdifficult to reproduce the blocks of the watermark-added contents datain the raster scanning order. In the event that illegal conduct removesthe watermark information from the watermark-added contents data, thejump-destination-address code words in the watermark information arealso lost. Thus, in this case, the blocks of the watermark-addedcontents data can not be reproduced in the raster scanning order.Accordingly, the watermark-added contents data are protected fromillegal reproduction.

Eighth Embodiment

An eighth embodiment of this invention is similar to the sixthembodiment thereof except for design changes mentioned hereafter. In theeighth embodiment of this invention, every frame (for example, everystill-picture frame) represented by watermark-added contents data isdivided into 12 sub-regions as shown in FIG. 21. For every frame, thewatermark-added contents data are divided into unit segments called“slices” corresponding to the 12 sub-regions respectively. For everyframe, the slices of the watermark-added contents data are arranged in araster scanning order as shown in FIG. 21.

In the eighth embodiment of this invention, the jump-destination addressgenerator 113 (see FIG. 14) produces a jump-destination address for eachslice. According to the produced jump-destination addresses, slices arerearranged in an order different from the raster scanning order. Thejump-destination address generator 113 converts every jump-designationaddress into a 3-bit code word, and feeds the 3-bit code word to thewatermark mixer 111 and the data rearranging device 115 (see FIG. 14).

The copyright information and the jump-destination address (the 3-bitcode word) fed to the watermark mixer 111 compose watermark information.The watermark mixer 111 embeds or inserts the watermark information intothe input contents data to get watermarked contents data, that is,watermark-added contents data. The watermark mixer 111 outputs thewatermark-added contents data to the MPEG encoder 114 (see FIG. 14). Thewatermark mixer 111 may directly output the watermark-added contentsdata to the data rearranging device 115 (see FIG. 14). In this case, theMPEG encoder 114 is omitted.

The MPEG encoder 114 compressively encodes the watermark-added contentsdata into watermark-added MPEG contents data according to a known MPEGencoding procedure. The MPEG encoder 114 outputs the watermark-addedMPEG contents data to the data rearranging device 115.

The jump-destination address generator 113 notifies the data rearrangingdevice 115 of a jump-designation address for each slice of thewatermark-added contents data outputted from the watermark mixer 111 orthe MPEG encoder 114. The data rearranging device 115 rearranges theslices of the watermark-added contents data from the watermark mixer 111or the MPEG encoder 114 in an order being different from the rasterscanning order and being determined by the jump-destination addresses.The data rearranging device 115 outputs the slices of thewatermark-added contents data to the data writer 116 (see FIG. 14) inthe rearranging-resultant order. Thus, the data rearranging device 115converts the watermark-added contents data into second watermark-addedcontents data. The data rearranging device 115 outputs the secondwatermark-added contents data to the data writer 116. The data writer116 changes the second watermark-added contents data into correspondingdata of a prescribed format suited for being recorded on a recordingmedium 117 (see FIG. 14). The data writer 116 records theprescribed-record-format contents data on the recording medium 117.

The eighth embodiment of this invention will be further described below.Pieces of contents data for every frame are rearranged in an orderdifferent from a raster scanning order on a unit-by-unit basis (forexample, a slice-by-slice basis) before being recorded on a recordingmedium. Jump-destination addresses relating to the data-piecerearrangement are embedded or inserted into the contents data asfragments of watermark information. For example, each slice isadditionally provided with a jump-destination address (a 3-bit codeword) representing the position of the head of a next slice which shouldfollow the present slice in the raster scanning order. During playback,jump-destination addresses are reproduced, and jumps between slices areexecuted in response to the reproduced jump-destination addresses toprovide a slice sequence accorded with the raster scanning order.

Slices are reproduced in the raster scanning order as shown in FIG. 21so that every 1-frame picture represented by the slices is normallyreproduced. Slices are rearranged in an order different from the rasterscanning order before being recorded. As a result, the 1-frame picturerepresented by the slices is scrambled. FIG. 22 shows an example of therearranging-resultant order which is caused by jumping some slices toprevious positions or later positions for a frame. The jump-based slicerearrangement keeps prescribed rules such that the first and last slicesfor every frame remain corresponding to the uppermost and lowermostsub-regions respectively.

A jump-destination address is represented by a word of a 3-bit code foreach slice. Every 3-bit code word indicates the type of a jump by whicha related slice can be returned to its correct position in the rasterscanning order. Specifically, a jump-destination-address code word of“000” indicates that a jump is unnecessary. A jump-destination-addresscode word of “001” indicates a jump by one slice in the downwarddirection. Here, the direction is defined with respect to correspondingsub-regions in a frame. A jump-destination-address code word of “010”indicates a jump by two slices in the downward direction. Ajump-destination-address code word of “011” indicates a jump by threeslices in the downward direction. A jump-destination-address code wordof “100” indicates a jump by one slice in the upward direction. Ajump-destination-address code word of “101” indicates a jump by twoslices in the upward direction. A jump-destination-address code word of“110” indicates a jump by three slices in the upward direction. Ajump-destination-address code word of “111” indicates a jump by fourslices in the upward direction.

Regarding the rearranging-resultant order in FIG. 22, for every frame,jump-destination-code words are added to the ends of the slicesrespectively. For example, a jump-destination-address code word of “001”is added to the end of the first block. A jump-destination-address codeword of “101” is added to the end of the second block.

In the eighth embodiment of this invention, watermark-added contentsdata stored in a recording medium 117 (see FIG. 18) have slices arrangedin an order different from a raster scanning order. The storedwatermark-added contents data include a set of audio information andvideo information placed in the slices. Alternatively, the storedwatermark-added contents data may include video information (forexample, still-picture information) placed in the slices. The storedwatermark-added contents data also include a jump-destination-addresscode word for every slice which indicates the type of a jump by whichthe related slice can be returned to its correct position in the rasterscanning order. The stored watermark-added contents data further includecopyright information. The jump-destination-address code words and thecopyright information compose watermark information.

At an initial stage of 1-frame reproduction of the watermark-addedcontents data from the recording medium 117, the data reader 121 (seeFIG. 18) reads out the first slice of the watermark-added contents datafrom the recording medium 117. The data reader 121 feeds the first sliceof the watermark-added contents data to the MPEG decoder 122 and thewatermark detecting device 123 (see FIG. 18). The data reader 121 maydirectly feed the first slice of the watermark-added contents data tothe display 124 (see FIG. 18). In this case, the MPEG decoder 122 isomitted. The watermark detecting device 123 extracts the watermarkinformation from the first slice of the watermark-added contents data.The extracted watermark information includes thejump-destination-address code word relating to the first slice. Thewatermark detecting device 123 feeds the jump-destination-address codeword to the jump-destination address calculator 125 (see FIG. 18). Thejump-destination address calculator 125 computes a jump-destinationaddress from the jump-destination-address code word. Thejump-designation address denotes the position to which thecurrently-accessed point on the recording medium 117 should be jumped,that is, the position of the head of a slice to be accessed next. Thejump-destination address calculator 125 notifies the data reader 121 ofthe computed jump-destination address. After the read-out of the firstslice of the watermark-added contents data from the recording medium 117has been completed, the data reader 121 jumps the currently-accessedpoint on the recording medium 117 to the position denoted by thejump-destination address and then starts reading out the second slice ofthe watermark-added contents data from the recording medium 117.

Similarly, the data reader 121 reads out the second and later slices ofthe watermark-added contents data from the recording medium 117. Thedata reader 121 feeds the second and later slices of the watermark-addedcontents data to the MPEG decoder 122 and the watermark detecting device123. In the absence of the MPEG decoder 122, the data reader 121directly feeds the second and later slices of the watermark-addedcontents data to the display 124. The watermark detecting device 123extracts the watermark information from the second and later slices ofthe watermark-added contents data. The extracted watermark informationincludes the jump-destination-address code words relating to the secondand later slices. The watermark detecting device 123 feeds thejump-destination-address code words to the jump-destination addresscalculator 125. The jump-destination address calculator 125 computes ajump-destination address from each of the jump-destination-address codewords. The jump-designation address denotes the position to which thecurrently-accessed point on the recording medium 117 should be jumped,that is, the position of the head of a slice to be accessed next. Thejump-destination address calculator 125 notifies the data reader 121 ofthe computed jump-destination addresses. After the read-out of thecurrent slice of the watermark-added contents data from the recordingmedium 117 has been completed, the data reader 121 jumps thecurrently-accessed point on the recording medium 117 to the positiondenoted by the jump-destination address and then starts reading out thenext slice of the watermark-added contents data from the recordingmedium 117.

The MPEG decoder 122 receives the slices of the watermark-added contentsdata from the data reader 121. The MPEG decoder 122 subjects thewatermark-added contents data to an MPEG decoding procedure, therebyreproducing a video signal and an audio signal. The MPEG decoder 122feeds the reproduced video signal to the display 124 (see FIG. 18). Thereproduced video signal is indicated on the display 124. The MPEGdecoder 122 feeds the reproduced audio signal to the loudspeaker 128(see FIG. 18). The reproduced audio signal is converted intocorresponding sounds by the loudspeaker 128.

In the absence of the MPEG decoder 122, the display 124 directlyreceives the slices of the watermark-added contents data from the datareader 121. The display 124 indicates a picture or pictures (forexample, a still picture or still pictures) represented by the slices ofthe watermark-added contents data.

As understood from the previous description, the currently-accessedpoint on the recording medium 117 is jumped in response to each of thejump-destination-address code words. The jumps of the currently-accessedpoint in response to the jump-destination-address code words enable theslices of the watermark-added contents data to be sequentiallyreproduced in the raster scanning order.

When the jump-destination-address code words fail to be recovered, it isdifficult to reproduce the slices of the watermark-added contents datain the raster scanning order. In the event that illegal conduct removesthe watermark information from the watermark-added contents data, thejump-destination-address code words in the watermark information arealso lost. Thus, in this case, the slices of the watermark-addedcontents data can not be reproduced in the raster scanning order.Accordingly, the watermark-added contents data are protected fromillegal reproduction.

Ninth Embodiment

A ninth embodiment of this invention is similar to one of the sixth,seventh, and eighth embodiments thereof except for design changesmentioned hereafter.

FIG. 23 shows an apparatus for embedding or inserting a watermark intocontents data according to the ninth embodiment of this invention. Theapparatus of FIG. 23 includes a transmitter 118 connected between thedata rearranging device 115 and a transmission path. The transmitter 118receives the rearranged contents data (the rearranged-resultantwatermark-added MPEG contents data) from the data rearranging device115. The transmitter 118 outputs the rearranged contents data to thetransmission path. The rearranging-resultant contents data propagatealong the transmission path. Preferably, the transmitter 118 implementsformat conversion of the rearranged contents data before outputting theresultant data to the transmission path.

The transmission path includes a communication network such as a wirecommunication network, a radio communication network, or the Internet.

FIG. 24 shows an apparatus for reproducing or extracting a watermarkfrom watermarked contents data (watermark-added contents data) accordingto the ninth embodiment of this invention. The apparatus of FIG. 24includes a receiver 130 and a data rearranging device 132 whichcorrespond to the data reader 121 in the apparatus of FIG. 18. Thereceiver 130 is connected between a transmission path and the datarearranging device 132. The data rearranging device 132 is connectedwith the MPEG decoder 122, the watermark detecting device 123, and thejump-destination address calculator 125. The device 130 receiveswatermark-added contents data from the transmission path. Originally,the watermark-added contents data are transmitted by, for example, theapparatus of FIG. 23. The receiver 130 feeds the receivedwatermark-added contents data to the data rearranging device 132.Preferably, the receiver 130 implements format conversion of thereceived watermark-added contents data before feeding the resultant datato the data rearranging device 132. The jump-destination addresscalculator 125 notifies the data rearranging device 132 of the computedjump-destination addresses. The data rearranging device 132 rearrangesthe watermark-added contents data in response to the jump-destinationaddresses, thereby making them into agreement with the original order.The data rearranging device 132 feeds the resultant watermark-addedcontents data to the MPEG decoder 122 and the watermark detecting device123. The data rearrangement by the data rearranging device 132 isinverse with respect to that by the data rearranging device 115 (seeFIG. 23).

The data rearranging device 132 includes a buffer memory and aread/write circuit. Under the control by the read/write circuit, thewatermark-added contents data outputted from the receiver 130 are storedin the buffer memory. Under the control by the read/write circuit, thewatermark-added contents data are read out from the buffer memory beforebeing fed to the MPEG decoder 122 and the watermark detecting device123. The read/write circuit controls the sequence of the read-out of thewatermark-added contents data from the buffer memory in response to thejump-destination addresses so that the watermark-added contents dataoutputted from the buffer memory will be in the original order.

1. An apparatus for embedding a watermark into contents data,comprising: address generating means for indicating a jump destinationfor each of prescribed unit portions of compressively encoded data, andthereby designating an arrangement order different from an order ofreproducing the prescribed unit portions and generating ajump-destination address which affects reproduction of contents data;information generating means for generating copyright information;mixing means for embedding the jump-destination address generated by theaddress generating means and the copyright information generated by theinformation generating means into input contents data as watermarkinformation to generate first watermark-added contents data; an encoderfor compressively encoding the first watermark-added contents datagenerated by the mixing means into second watermark-added contents data;and rearranging means for rearranging prescribed unit portions of thesecond watermark-added contents data generated by the encoder inresponse to the jump-destination address generated by the addressgenerating means into an order different from the order of reproducingthe prescribed unit portions; wherein: the address generating meansincludes means for indicating a jump destination for each of cells ofcompressively encoded data, and thereby designating an arrangement timeorder different from a time order of reproducing the cells andgenerating a jump-destination address which affects reproduction ofcontents data; the mixing means includes means for embedding thejump-destination address generated by the means included in the addressgenerating means and the copyright information generated by theinformation generating means into the input contents data as thewatermark information to generate the first watermark-added contentsdata; and the rearranging means includes means for rearranging cells ofthe second watermark-added contents data generated by the encoder inresponse to the jump-destination address generated by the means includedin the address generating means into a time order different from thetime order of reproducing the cells.
 2. An apparatus for embedding awatermark into contents data, comprising: address generating means forindicating a jump destination for each of prescribed unit portions ofcompressively encoded data, and thereby designating an arrangement orderdifferent from an order of reproducing the prescribed unit portions andgenerating a jump-destination address which affects reproduction ofcontents data, wherein the prescribed unit portions correspond to oneframe represented by the compressively encoded data; informationgenerating means for generating copyright information; mixing means forembedding the jump-destination address generated by the addressgenerating means and the copyright information generated by theinformation generating means into input contents data as watermarkinformation to generate first watermark-added contents data; an encoderfor compressively encoding the first watermark-added contents datagenerated by the mixing means into second watermark-added contents data;and rearranging means for rearranging prescribed unit portions of thesecond watermark-added contents data generated by the encoder inresponse to the jump-destination address generated by the addressgenerating means into an order different from the order of reproducingthe prescribed unit portions; wherein: the address generating meansincludes means for indicating a jump destination for each of cells ofcompressively encoded data, and thereby designating an arrangement timeorder different from a time order of reproducing the cells andgenerating a jump-destination address which affects reproduction ofcontents data; the mixing means includes means for embedding thejump-destination address generated by the means included in the addressgenerating means and the copyright information generated by theinformation generating means into the input contents data as thewatermark information to generate the first watermark-added contentsdata; and the rearranging means comprises means for rearranging cells ofthe second watermark-added contents data generated by the encoder inresponse to the jump-destination address generated by the means includedin the address generating means into a time order different from thetime order of reproducing the cells.
 3. An apparatus for reproducingoriginal contents data from watermarked contents data generated by (1)indicating a jump destination for each of prescribed unit portions ofcompressively encoded data, and thereby designating an arrangement orderdifferent from an order of reproducing the prescribed unit portions andgenerating a jump-destination address which affects reproduction ofcontents data, (2) generating copyright information, (3) embedding thejump-destination address and the copyright information into inputcontents data as watermark information to generate first watermark-addedcontents data, (4) compressively encoding the first watermark-addedcontents data into second watermark-added contents data, and (5)rearranging prescribed unit portions of the second watermark-addedcontents data in response to the jump-destination address into an orderdifferent from the order of reproducing the prescribed unit portions,the apparatus comprising: data reading means for reading inputwatermarked encoded contents data; watermark information detecting meansfor detecting a jump-destination address from the input watermarkedencoded contents data read by the data reading means; addresscalculating means for calculating a data reading address for each of theprescribed unit portions from the jump-destination address detected bythe watermark information detecting means, and outputting the calculatedaddress to the data reading means and thereby enabling the data readingmeans to read the prescribed unit portions of the input watermarkedencoded contents data in an original normal order; and a decoder fordecoding the input watermarked encoded contents data read by the datareading means into watermarked decoded contents data; wherein thewatermarked contents data is generated by (1) indicating a jumpdestination for each of cells of the compressively encoded data, andthereby designating an arrangement time order different from a timeorder of reproducing the cells and generating a jump-destination addresswhich affects reproduction of contents data, (2) generating thecopyright information, (3) embedding said generated jump-destinationaddress and said generated copyright information into input contentsdata as watermark information to generate the first watermark-addedcontents data, (4) compressively encoding the first watermark-addedcontents data into the second watermark-added contents data, and (5)rearranging cells of the second watermark-added contents data inresponse to said generated jump-destination address into a time orderdifferent from the time order of reproducing the cells.
 4. An apparatusfor reproducing original contents data from watermarked contents datagenerated by (1) indicating a jump destination for each of prescribedunit portions of compressively encoded data, and thereby designating anarrangement order different from an order of reproducing the prescribedunit portions and generating a jump-destination address which affectsreproduction of contents data, wherein the prescribed unit portionscorrespond to one frame represented by the compressively encoded data,(2) generating copyright information, (3) embedding the jump-destinationaddress and the copyright information into input contents data aswatermark information to generate first watermark-added contents data,(4) compressively encoding the first watermark-added contents data intosecond watermark-added contents data, and (5) rearranging prescribedunit portions of the second watermark-added contents data in response tothe jump-destination address into an order different from the order ofreproducing the prescribed unit portions, the apparatus comprising: datareading means for reading input watermarked encoded contents data;watermark information detecting means for detecting a jump-destinationaddress from the input watermarked encoded contents data read by thedata reading means; address calculating means for calculating a datareading address for each of the prescribed unit portions from thejump-destination address detected by the watermark information detectingmeans, and outputting the calculated address to the data reading meansand thereby enabling the data reading means to read the prescribed unitportions of the input watermarked encoded contents data in an originalnormal order; and a decoder for decoding the input watermarked encodedcontents data read by the data reading means into watermarked decodedcontents data; wherein the watermarked contents data is generated by (1)indicating a jump destination for each of cells of the compressivelyencoded data, and thereby designating an arrangement time orderdifferent from a time order of reproducing the cells and generating ajump-destination address which affects reproduction of contents data,(2) generating the copyright information, (3) embedding said generatedjump-destination address and said generated copyright information intoinput contents data as watermark information to generate the firstwatermark-added contents data, (4) compressively encoding the firstwatermark-added contents data into the second watermark-added contentsdata, and (5) rearranging cells of the second watermark-added contentsdata in response to said generated jump-destination address into a timeorder different from the time order of reproducing the cells.
 5. Arecording medium for storing watermarked contents data generated by (1)indicating a jump destination for each of prescribed unit portions ofcompressively encoded data, and thereby designating an arrangement orderdifferent from an order of reproducing the prescribed unit portions andgenerating a jump-destination address which affects reproduction ofcontents data, (2) embedding the jump-destination address and copyrightinformation into input contents data as watermark information togenerate first watermark-added contents data, (3) compressively encodingthe first watermark-added contents data into second watermark-addedcontents data, and (4) rearranging prescribed unit portions of thesecond watermark-added contents data in response to the jump-destinationaddress into an order different from the order of reproducing theprescribed unit portions; wherein the watermarked contents data isgenerated by (1) indicating a jump destination for each of cells of thecompressively encoded data, and thereby designating an arrangement timeorder different from a time order of reproducing the cells andgenerating a jump-destination address which affects reproduction ofcontents data, (2) embedding said generated jump-destination address andthe copyright information into input contents data as watermarkinformation to generate the first watermark-added contents data, (3)compressively encoding the first watermark-added contents data into thesecond watermark-added contents data, and (4) rearranging cells of thesecond watermark-added contents data in response to said generatedjump-destination address into a time order different from the time orderof reproducing the cells.
 6. A recording medium for storing watermarkedcontents data generated by (1) indicating a jump destination for each ofprescribed unit portions of compressively encoded data, and therebydesignating an arrangement order different from an order of reproducingthe prescribed unit portions and generating a jump-destination addresswhich affects reproduction of contents data, wherein the prescribed unitportions correspond to one frame represented by the compressivelyencoded data, (2) embedding the jump-destination address and copyrightinformation into input contents data as watermark information togenerate first watermark-added contents data, (3) compressively encodingthe first watermark-added contents data into second watermark-addedcontents data, and (4) rearranging prescribed unit portions of thesecond watermark-added contents data in response to the jump-destinationaddress into an order different from the order of reproducing theprescribed unit portions; wherein the watermarked contents data isgenerated by (1) indicating a jump destination for each of cells of thecompressively encoded data, and thereby designating an arrangement timeorder different from a time order of reproducing the cells andgenerating a jump-destination address which affects reproduction ofcontents data, (2) embedding said generated jump-destination address andthe copyright information into input contents data as watermarkinformation to generate the first watermark-added contents data, (3)compressively encoding the first watermark-added contents data into thesecond watermark-added contents data, and (4) rearranging cells of thesecond watermark-added contents data in response to said generatedjump-destination address into a time order different from the time orderof reproducing the cells.